fix(proxy): pass-through multipart uploads and Bedrock JSON body by shivamrawat1 · Pull Request #25464 · BerriAI/litellm

shivamrawat1 · 2026-04-10T02:48:45Z

Relevant issues

Pass-through routes now forward multipart/form-data (e.g. file uploads) correctly. Programmatic Bedrock-style callers still supply a pre-built JSON body via request.state instead of a route parameter. Streaming multipart uses build_request + send(stream=True) for httpx 0.28. Adds/updates unit tests for multipart forwarding and forward_multipart.

Cause
custom_body: Optional[dict] on the pass-through handler — FastAPI treated it as the HTTP JSON body, so multipart was validated as JSON and failed before the handler ran (often surfacing as RequestValidationError / UTF-8 errors on binary payloads).
not _parsed_body gating multipart — After injecting litellm_logging_obj, _parsed_body was never empty, so the code took json=_parsed_body instead of rebuilding multipart.
AsyncClient.request(..., stream=...) — In httpx 0.28, request() does not accept stream; streaming must use send(..., stream=True) after build_request.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

50-55 passing tests: main is stable with minor issues.

45-49 passing tests: acceptable but needs attention

<= 40 passing tests: unstable; be careful with your merges and assess the risk.

Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:

Type

🐛 Bug Fix
✅ Test

Changes

Fix
Remove body-parameter custom_body from pass-through handlers; read optional programmatic JSON from request.state[LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY] and clear it in finally. Bedrock sets that attribute before calling the handler.
Pass forward_multipart=is_multipart into non_streaming_http_request_handler and branch on that (with is_multipart still meaning “multipart request and no JSON envelope custom_body”).
In make_multipart_http_request, if stream is true, use build_request + send(stream=True); otherwise keep request(...) without stream.
Add/update tests for the above paths.

- Route multipart forwarding on forward_multipart instead of empty _parsed_body so litellm_logging_obj no longer forces json= for file uploads. - Remove custom_body from pass-through endpoint signatures; FastAPI treated it as a JSON body and rejected multipart before the handler ran. Bedrock passes JSON via request.state (LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY). - Use build_request + send(stream=True) for streaming multipart; httpx 0.28 AsyncClient.request does not accept stream=. - Add regression test for non-empty _parsed_body multipart path; update Bedrock custom-body test and query-params test for forward_multipart. Made-with: Cursor

vercel · 2026-04-10T02:48:50Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Apr 11, 2026 10:28pm

codspeed-hq · 2026-04-10T02:50:19Z

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing litellm_passthrough_contenttype (3742b0c) with main (21e4071)}

shivamrawat1 · 2026-04-10T02:51:57Z

@greptile review

greptile-apps · 2026-04-10T02:58:09Z

Greptile Summary

This PR fixes three root causes behind broken multipart/form-data pass-through and Bedrock JSON forwarding: (1) removing custom_body: Optional[dict] from the FastAPI handler signature so multipart requests are no longer rejected as JSON before the handler runs, (2) replacing the not _parsed_body gate (which was always truthy after litellm_logging_obj injection) with an explicit forward_multipart flag, and (3) using build_request + send(stream=True) for streaming multipart to comply with httpx 0.28.

Confidence Score: 5/5

PR is safe to merge — three clearly-scoped root-cause fixes with corresponding mock tests and no new P0/P1 issues.

All remaining findings are P2 or lower. The logic for is_multipart/forward_multipart, the state-based body injection, and the httpx 0.28 streaming fix are all correct. The constant is now properly imported in both proxy files, resolving the NameError flagged in a prior review. Tests are mock-only and add meaningful regression coverage.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/types/passthrough_endpoints/pass_through_endpoints.py	Adds `LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY` constant to the types module, giving both proxy files a single import source and avoiding circular imports.
litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py	Replaces `custom_body=data` kwarg on `endpoint_func` with `setattr(request.state, LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY, data)` before calling the endpoint; imports the constant correctly from the types module.
litellm/proxy/pass_through_endpoints/pass_through_endpoints.py	Core logic changes: adds explicit `forward_multipart` flag to `non_streaming_http_request_handler`, adds `stream` param to `make_multipart_http_request` (uses `build_request+send` path), handles streaming multipart in `pass_through_request`, and reads/cleans up `request.state` key in `create_pass_through_route`.
tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py	Adds regression test for multipart with non-empty `_parsed_body`, updates `test_create_pass_through_route_custom_body_url_target` to use `request.state` injection, and adds `SimpleNamespace()` state to tests that previously lacked it.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[bedrock_proxy_route] -->|setattr request.state| B[endpoint_func via create_pass_through_route]
    B -->|reads state_custom_body| C{state_custom_body?}
    C -->|Yes - dict| D[final_custom_body = state_custom_body]
    C -->|No| E{custom_body_data from request?}
    E -->|Yes| F[final_custom_body = custom_body_data]
    E -->|No| G[final_custom_body = None]
    D & F & G --> H[pass_through_request custom_body=final_custom_body]
    H --> I{is_multipart? = is_multipart_request AND NOT custom_body}
    I -->|True| J{stream?}
    I -->|False| K{stream?}
    J -->|Yes| L[make_multipart_http_request build_request + send stream=True]
    J -->|No| M[non_streaming_http_request_handler forward_multipart=True]
    K -->|Yes| N[build_request + send stream=True]
    K -->|No| O[non_streaming_http_request_handler forward_multipart=False json=_parsed_body]
    B -->|finally: delattr state key| P[cleanup request.state]

_{Reviews (5): Last reviewed commit: "refactor: define pass-through custom bod..." | Re-trigger Greptile}

greptile-apps · 2026-04-10T02:58:13Z

+        if stream:
+            req = async_client.build_request(
+                request.method,
+                url,
+                headers=headers_copy,
+                params=requested_query_params,
+                files=files,
+                data=form_data_dict,
+            )
+            return await async_client.send(req, stream=True)


Missing test for streaming multipart path

The stream=True branch in make_multipart_http_request (build_request + send(stream=True)) is new code introduced by this PR, but none of the existing unit tests exercise it. Both test_make_multipart_http_request and test_make_multipart_http_request_removes_content_type_header only mock async_client.request and never pass stream=True, so the build_request/send path is entirely untested. A dedicated test should mock async_client.build_request and async_client.send to verify headers, files, and the stream=True argument are forwarded correctly.

Restores Windows-style line endings to match main/origin main for this file, removing the full-file noise diff from an accidental LF-only normalization. Made-with: Cursor

…import' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

greptile-apps · 2026-04-11T21:49:08Z

        is_streaming_request=is_streaming_request,
        _forward_headers=True,
    )  # dynamically construct pass-through endpoint based on incoming path
+    setattr(request.state, LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY, data)


NameError — LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY is not in scope

LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY is defined in pass_through_endpoints.py but is not imported into this module. The top-level import block (lines 38-43) brings in HttpPassThroughEndpointHelpers, create_pass_through_route, etc., but not this constant. _get_litellm_pass_through_custom_body_state_key() was added to provide a lazy import, but it is never called here — the bare name is referenced instead. Every call to bedrock_proxy_route will raise NameError: name 'LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY' is not defined.

Fix: add the constant to the existing top-level import (no circular-import concern — llm_passthrough_endpoints already imports from pass_through_endpoints at module level):

from litellm.proxy.pass_through_endpoints.pass_through_endpoints import ( HttpPassThroughEndpointHelpers, LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY, create_pass_through_route, create_websocket_passthrough_route, websocket_passthrough_request, )

The _get_litellm_pass_through_custom_body_state_key() helper can then be removed as it becomes dead code.

…ssthrough Fixes NameError when bedrock_proxy_route sets custom body on request.state. Remove unused lazy-loader helper. Made-with: Cursor

shivamrawat1 · 2026-04-11T22:08:50Z

@greptile review again with new commit that resolves p0

Avoid module-level cyclic import between llm_passthrough_endpoints and pass_through_endpoints; CodeQL and partial init order no longer risk undefined LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY. Made-with: Cursor

shivamrawat1 · 2026-04-11T22:27:56Z

@greptile review with new commit that remove cyclic import

shivamrawat1 changed the title ~~fix(proxy): pass-through multipart uploads and Bedrock custom body~~ fix(proxy): pass-through multipart uploads and Bedrock JSON body Apr 10, 2026

greptile-apps bot reviewed Apr 10, 2026

View reviewed changes

chore(proxy): match CRLF line endings in pass_through_endpoints

20b16b3

Restores Windows-style line endings to match main/origin main for this file, removing the full-file noise diff from an accidental LF-only normalization. Made-with: Cursor

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 21:25 — with GitHub Actions Inactive

shivamrawat1 had a problem deploying to integration-postgres April 11, 2026 21:25 — with GitHub Actions Error

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 21:25 — with GitHub Actions Inactive

vercel bot deployed to Preview April 11, 2026 21:26 View deployment

github-advanced-security AI found potential problems Apr 11, 2026

View reviewed changes

Comment thread litellm/proxy/pass_through_endpoints/llm_passthrough_endpoints.py Fixed

Potential fix for pull request finding 'CodeQL / Module-level cyclic …

627d70a

…import' Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 21:46 — with GitHub Actions Inactive

shivamrawat1 had a problem deploying to integration-postgres April 11, 2026 21:46 — with GitHub Actions Error

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 21:46 — with GitHub Actions Inactive

shivamrawat1 assigned shivamrawat1 and yuneng-berri and unassigned shivamrawat1 Apr 11, 2026

vercel bot deployed to Preview April 11, 2026 21:47 View deployment

greptile-apps bot reviewed Apr 11, 2026

View reviewed changes

fix: import LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY for bedrock pa…

574eb20

…ssthrough Fixes NameError when bedrock_proxy_route sets custom body on request.state. Remove unused lazy-loader helper. Made-with: Cursor

shivamrawat1 had a problem deploying to integration-postgres April 11, 2026 22:08 — with GitHub Actions Error

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 22:08 — with GitHub Actions Inactive

vercel bot deployed to Preview April 11, 2026 22:09 View deployment

refactor: define pass-through custom body state key in types module

3742b0c

Avoid module-level cyclic import between llm_passthrough_endpoints and pass_through_endpoints; CodeQL and partial init order no longer risk undefined LITELLM_PASS_THROUGH_CUSTOM_BODY_STATE_KEY. Made-with: Cursor

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 22:26 — with GitHub Actions Inactive

shivamrawat1 temporarily deployed to integration-postgres April 11, 2026 22:27 — with GitHub Actions Inactive

shivamrawat1 had a problem deploying to integration-postgres April 11, 2026 22:27 — with GitHub Actions Error

vercel bot deployed to Preview April 11, 2026 22:28 View deployment

yuneng-berri self-requested a review April 11, 2026 22:39

yuneng-berri approved these changes Apr 11, 2026

View reviewed changes

yuneng-berri merged commit 9a43e32 into main Apr 11, 2026
100 of 105 checks passed

yuneng-berri deleted the litellm_passthrough_contenttype branch April 11, 2026 22:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(proxy): pass-through multipart uploads and Bedrock JSON body#25464

fix(proxy): pass-through multipart uploads and Bedrock JSON body#25464
yuneng-berri merged 5 commits intomainfrom
litellm_passthrough_contenttype

shivamrawat1 commented Apr 10, 2026 •

edited

Loading

Uh oh!

vercel bot commented Apr 10, 2026 •

edited

Loading

Uh oh!

codspeed-hq bot commented Apr 10, 2026 •

edited

Loading

Uh oh!

shivamrawat1 commented Apr 10, 2026

Uh oh!

greptile-apps bot commented Apr 10, 2026 •

edited

Loading

Important Files Changed

Uh oh!

greptile-apps bot Apr 10, 2026

Uh oh!

Uh oh!

greptile-apps bot Apr 11, 2026

Uh oh!

shivamrawat1 commented Apr 11, 2026

Uh oh!

shivamrawat1 commented Apr 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shivamrawat1 commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Relevant issues

Pre-Submission checklist

Delays in PR merge?

CI (LiteLLM team)

Type

Changes

Uh oh!

vercel bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

shivamrawat1 commented Apr 10, 2026

Uh oh!

greptile-apps bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps bot Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

greptile-apps bot Apr 11, 2026

Choose a reason for hiding this comment

Uh oh!

shivamrawat1 commented Apr 11, 2026

Uh oh!

shivamrawat1 commented Apr 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

shivamrawat1 commented Apr 10, 2026 •

edited

Loading

vercel bot commented Apr 10, 2026 •

edited

Loading

codspeed-hq bot commented Apr 10, 2026 •

edited

Loading

greptile-apps bot commented Apr 10, 2026 •

edited

Loading